Additional functionality for telephone numbers and utilization of context information associated with telephone numbers in computer documents

ABSTRACT

Telephone numbers are identified and highlighted on any computer document being loaded into a graphical user interface (GUI) for viewing by a user, and context information associated with the telephone numbers so tagged may also be viewed. The user may click on any particular highlighted telephone number or context information in the computer document while displayed in the GUI window in order to store the telephone number together with any desired context information associated therewith, or perform other functions, such as sending a facsimile transmission to the number, sending a message (e.g., SMS, IM or e-mail) or searching an associated database for prior occurrences of the telephone number or context information.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation-in-part of U.S. patent applicationSer. No. 10/697,468 which was filed with the U.S. Patent and TrademarkOffice on Oct. 30, 2003, Ser. No. 11/191,700, which was filed with theU.S. Patent and Trademark Office on Jul. 28, 2005 and Ser. No.11/224,160, which was filed with the U.S. Patent and Trademark Office onSep. 12, 2005, and which are each hereby incorporated by reference intheir entireties.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present disclosure relates generally to computer documents and,specifically, to providing users with additional functionality fortelephone numbers and context information surrounding telephone numbersfound in computer documents.

2. Description of the Related Art

Although computers (i.e., devices that compute) date back to the abacusin Ancient China and Babylonia, it has only been in the last centurythat the term “computer” has developed its modern definition as amachine that accepts data and manipulates it to produce a result basedon a program (sequence of instructions). In recent years, astechnological growth and innovations rapidly bring new devices andappliances to the world, the technology of computing has become more andmore vitally connected to the world's economy and people's daily life,as seen in the evolving form and mutations of the computer: from PDAs,notebooks, personal computers, minicomputers, LAN/Internet servers, andmainframes to cell phones and embedded processors in householdappliances, car components, pets, furniture, etc. Today's computer canbe any electronic, magnetic, optical, or software device for processingdata, and such data processing can take the form of logical, arithmetic,memory, and/or storage functions.

As the computer has evolved in the last century, so have the dataobjects which the computer manipulates. In this application, such dataobjects will be referred to as “computer documents”. Computer documentsinclude, for example, word processing files (such as an MS Worddocument), markup language documents (such as HTML or XML files),database records, graphics/image files, and audio/video files.

Furthermore, the technology of computing and the technology ofcommunication has been going through a process of merging—a process inwhich the distinctions between the technologies of the telephone, thetelevision, the personal computer, the Internet, and the cellular phoneare increasingly blurred, if not meaningless. The functionalities ofwhat were once separate devices are now freely shared between and amongdevices. One's cellular phone can surf the Internet, while one'spersonal computer (PC) can make telephone calls.

In U.S. patent application Ser. No. 10/697,468 (“the '468 application”),from which the present application claims priority, the present inventordisclosed a method by which web browsers are provided with thecapability and GUI functionality to act upon and use telephone numberswhich appear on web pages. While a web page is being downloaded, it isparsed to determine whether there are any telephone numbers present onthe web page, and all found instances of telephone numbers arehighlighted when the web page was rendered, thereby bringing thetelephone numbers to the user's attention, and indicating the additionalfunctionality available for those highlighted numbers. In the specificimplementation described in the '468 application, the user couldleft-click a highlighted telephone number to call the number usingtelephone connection software such as Avaya's IP-phone. Ifright-clicked, an option menu would appear, allowing the user to selectone of a group of actions to perform on the telephone number (such assaving the telephone number in memory, or forwarding it to an addressbook or another user, etc.).

The '468 application was directed to markup language documents, i.e.,computer files that are rendered, accessed, and/or manipulated by amarkup language document browser. However, such a telephone numbercapability and GUI functionality is desirable for all types of computerdocuments, not merely markup language documents. For example, whenreading an e-mail or a word processing document, or reviewing aspreadsheet, the capability and GUI functionality of recognizing,highlighting, and acting upon any instances of telephone numbers wouldbe both practical and useful for the user.

It would also be useful if the telephone number could be saved, forfuture use. In this case, it would also be useful if the telephonenumber could be saved with some information to give context to thetelephone number, such as a name or other identification associated withthe telephone number, to enhance the value of the telephone numberitself.

SUMMARY OF THE INVENTION

One object of the present invention is to provide a method foridentifying, highlighting, and providing the capability to act upon, allinstances of telephone numbers in any type of computer document,together with information which provides context for the telephonenumber, in which the action includes the ability to schedule acommunication session with the telephone number, either immediately orat a later time; send a facsimile transmission to the number (if it is afax number); send a short message service (SMS) message or instantmessage (IM) to the number; or send the telephone number directly to aseparate device (such as a telephone or PDA) via suitable means (such asIrDA or Bluetooth transmission) or to multiple devices (as, for example,by a WAP-push).

Another object of the present invention is to provide a method for auser to act upon more than one uncovered telephone numbersimultaneously.

There are many other objects besides the few listed above, as will bemade explicitly and/or implicitly clear to one of ordinary skill in theart when reading this specification and the claims appended hereto.

These and other objects are substantially achieved by the presentlypreferred embodiments of the invention, which provide a method for (a)automatically detecting any telephone number present in any computerdocument (e.g., a Microsoft Word file, a Microsoft Excel spreadsheet, aMicrosoft Power Point presentation, a Microsoft Outlook email, etc.);(b) highlighting any detected telephone numbers in the computer documentso as to indicate their presence to the user, and to indicate thatfunctionality is available for the highlighted telephone number; and (c)performing one or more desired operations upon any highlighted telephonenumber with only one or more actions on the viewer/user's part, wheresuch functionality is not necessarily native to said computer document.Such desired operations may include: storing the telephone numbertogether with context information to provide added meaning to thetelephone number; transmitting the telephone number, together with thecontext information, to a designated piece of hardware or software inorder that the designated piece of hardware/software perform a desiredaction either with or upon the telephone number; the ability to schedulea communication session with the telephone number, either immediately orat a later time; send a facsimile transmission to the number (if it is afax number); send a short message service (SMS) message or instantmessage (IM) to the number; or send the telephone number directly to aseparate device (such as a telephone or PDA) via suitable means (such asIrDA or Bluetooth transmission) or to multiple devices (as, for example,by a WAP-push) and providing the user with a menu of options for whataction to perform with or upon the detected instances and the contextinformation.

It should be understood that the steps in the methods described hereinmay be performed in either one location or many locations, by either onedevice or many devices, and that different devices may perform differentsteps. Of course, the order of the steps may also change, depending onthe particular embodiment.

Other objects and features of the present invention will become apparentfrom the following detailed description considered in conjunction withthe accompanying drawings. It is to be understood, however, that thedrawings are designed solely for purposes of illustration and not as adefinition of the limits of the invention, for which reference should bemade to the appended claims. It should be further understood that thedrawings are not necessarily drawn to scale and that, unless otherwiseindicated, they are merely intended to conceptually illustrate thestructures and procedures described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, where like numerals indicate like objects, and likedigits within different drawings indicate similarity, but not identity,between objects/components, where:

FIG. 1A is a flowchart of a method for automatically detecting,highlighting, and providing functionality for, telephone numbers in acomputer document according to a general conceptual embodiment of thepresent invention;

FIG. 1B is a flowchart of a specific implementation of steps 150 and 160in FIG. 1A according to a general conceptual embodiment of the presentinvention;

FIG. 1C is a flowchart of a method for automatically detecting,highlighting, and providing functionality for, telephone numbers in acomputer document, wherein the integrity of the original computerdocument is maintained. according to another general conceptualembodiment of the present invention;

FIG. 1D is a representation of a hierarchical tree of information in acomputer document according to a preferred embodiment of the presentinvention;

FIG. 2A is an exemplary illustration of a web page in a web browser GUIhaving highlighted telephone numbers with provided functionalityaccording to a first preferred embodiment of the present invention;

FIG. 2B is a schematic diagram of the components which identify,highlight, and provide functionality for, telephone numbers in a markuplanguage document, such as is seen in the example of FIG. 2A, accordingto a first embodiment of the present invention;

FIG. 3A is an exemplary illustration of a computer document in a wordprocessing GUI having highlighted telephone numbers with providedfunctionality according to a second preferred embodiment of the presentinvention;

FIG. 3B is a schematic diagram of the components which identify,highlight, and provide functionality for, telephone numbers in acomputer document, such as is seen in the exemplary word processingdocument of FIG. 3A, according to a second preferred embodiment of thepresent invention;

FIG. 4A is an exemplary illustration of a computer document in an e-mailGUI having highlighted telephone numbers with provided functionalityaccording to a third preferred embodiment of the present invention; and

FIG. 4B is a schematic diagram of the components which identify,highlight, and provide functionality for, telephone numbers in acomputer document, such as is seen in the exemplary e-mail of FIG. 4A,according to a third preferred embodiment of the present invention.

DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED EMBODIMENTS

The method according to the presently preferred embodiments will begenerally, i.e., conceptually, described with reference to theflowcharts in FIGS. 1A-1B-1C, while three preferred embodiments will bedescribed in reference to the exemplary screen shots and schematicdiagrams of FIGS. 2A-2B (for a computer document such as a markuplanguage document), FIGS. 3A-3B (for a computer document such as a wordprocessing file), and FIGS. 4A-4B (for a computer document such as ane-mail).

FIG. 1A is a general schematic flowchart of the conceptual steps in thepresently preferred embodiments; thus, the steps therein may beperformed in a different order, or some steps may be combined or asingle step separated into two or more sub-steps. In other words, thepresent invention is not intended to be limited to the order, thenumber, or overall structure of steps in FIG. 1A (or FIGS. 1B-1C, forthat matter). Furthermore, the appearance of the exemplary computerdocuments in FIGS. 2A-3A-4A or the particular structures depicted in theschematic diagrams of FIGS. 2B-3B-4B are not intended to limit thepresent invention in any way, shape, or form. Although the threepresently preferred embodiments are referred to as “first”, “second”,and “third” herein, it is not intended as any sort of limitation on thenumber and variety of possible embodiments according to the presentinvention.

FIG. 1B is a flowchart showing some of the possible sub-steps comprisingtwo specific steps in FIG. 1A.

FIG. 1C is a general schematic flowchart, like FIG. 1A, with additionalsteps for embodiments in which it is important to keep the integrity ofthe computer document intact. In such embodiments, it is desirable to beable to return the computer document to its original format before anyparticular “file action” is performed on the computer document. “Fileactions” include, for example, saving, forwarding, copying, or any otheroperation where the original format of the computer document is needed(or desired).

In the presently preferred embodiments of the present invention,telephone numbers are detected in a computer document being opened in anapplication, any detected telephone numbers are highlighted in theopened computer document in order to indicate to the user thatadditional functionality is available for the detected telephonenumbers, and the document user/viewer may indicate that certainaction(s) upon or with the telephone numbers is/are desired by theuser/viewer. In the presently preferred embodiments, the user indicatesthe desired action(s) by using an input device, such as a computermouse, and a visual representation in the GUI, such as an arrow cursor.Although the presently preferred embodiments are described in terms of acomputer mouse, any input device capable of selecting an item on thecomputer screen may be used by the user, for example, a touchpad, akeyboard, a joystick, voice command, etc., as would be understood by oneskilled in the art.

It should be understood that the method according to the presentlypreferred embodiments uses the functionality provided by the applicationto which the computer document is native. For example, for a wordprocessing document, the word processing application provides thewindows or GUIs through which the computer documents are being viewed.In other words, this invention is not directed to the creation of a newGUI, but rather using the functionality provided by the native GUI inthe application, as will be made clear in the detailed descriptionsbelow.

In step 100 of FIG. 1A, a computer document is being loaded or opened bythe appropriate application for that type of computer document (e.g, aword processing program for a text document, an e-mail program for ane-mail).

In step 110, the contents of the computer document are parsed, i.e., thecontent of the document is broken down and analyzed so that possibletelephone numbers are detected. When the computer document is beingparsed, it is examined for information which may be context informationfor any telephone number in the computer document. This is accomplishedin step 115, in which the computer document being loaded is examined forinformation which would permit the later identification of contextinformation, such as, for example, the type of computer document beingloaded (e.g., web page, e-mail message, word processing document, orspreadsheet) the location of the computer document (e.g., the URL of aweb page, the storage location of an e-mail message, word processingdocument or spreadsheet) and any other information which may bedesirable to establish the context of the telephone number, such as, forexample, the sender, recipient and/or subject line of an e-mail messagein which the telephone number was found. Depending upon the application,and the needs of the user, other types and locations of contextinformation may be of interest, and one of ordinary skill in the artwould be able to ascertain that information depending upon the needs ofa particular application. When parsing the computer document, the entirecontents of the computer document are analyzed. The contents of thecomputer document which are associated with the telephone numbers maycontain context information which is of interest. To make thisdetermination in some applications, it may be desirable to establish aninformation structure, such as a relational database or a hierarchicalvirtual tree arrangement, of the information found in the computerdocument, so that when it is determined that a telephone number has beenfound, the context information may be identified and stored. Ahierarchical tree containing such information may be seen in FIG. 1D,generally at 20, in which various data points 22-40 are shown withlogical connections 42-58 establishing the relationships between thevarious data points. Each data point may be referred to as a “node” andmay be a point of origin for the tree (a “root”) such as node 22, aterminal point of a branch (a “leaf”) such as nodes 26, 34, 36, 40 and42, or a branching point such as nodes 24, 28, 30 and 36 which connectsto other branching points or leaves. Each node contains data and isassociated with other nodes as shown in the tree structure. The mappingof such a tree is well-known on the art and so will not be described infurther detail.

In step 120, the possible telephone numbers detected in step 110 areclassified in order to identify the most likely telephone numbers in thegroup of possible telephone numbers (this will be described in greaterdetail below). In step 130, tags (which are not seen by the user) areplaced around each telephone number identified in step 120. When thecomputer document is visually rendered in step 140, these tags willcause the identified telephone numbers to be highlighted in the computerdocument and also give these highlighted telephone numbers additionalfunctionality (as will also be described in greater detail below). Inthis context, the term “highlight” denotes any form of differentiatingan object on the presented computer document, most likely by changingthe object's appearance, e.g. yellow highlighting, underlining, bolding,etc.

In step 150, the method awaits for an indication that the user hasinteracted with a highlighted telephone number in the visually renderedcomputer document. If an event is detected which indicates that the userhas interacted with a highlighted telephone number, specifically, if theuser has indicated a desired operation, the desired operation indicatedby the user is performed in step 160. At any time during the method inFIG. 1A, the user may jump to another application (not shown), therebyopening a computer document and initiating the process at step 100 forthat application.

FIG. 1B provides further details of a specific implementation of steps150 and 160 in FIG. 1A. In step 151, it is determined whether the user'scursor is located over a highlighted telephone number in the computerdocument as visually rendered in the GUI. If the cursor is located overa telephone number, the appearance of the cursor changes in order tofurther indicate the available functionality for the highlightedtelephone number in step 152. For example, the cursor icon could changeto a telephone icon. Even in this specific implementation, step 152 isoptional, as shown by the dotted lines forming the box in FIG. 1B.

In step 153, it is determined whether the user has selected thehighlighted telephone number. In the specific implementation of FIG. 1B,the user's input device is a computer mouse with a left-hand and aright-hand button, whereby the user can place the cursor over a specificarea on a computer screen and depressing either the left-hand button fora “left-click” or a right-hand button for a “right-click”. In otherimplementations, there may be only one button on the computer mouse,and, instead of a right-click and a left-click, the user may depress thesingle mouse button either once or twice in rapid succession to indicatedifferent choices. In less preferred embodiments, the user may have ameans that can only indicate a single type of selection.

In the specific implementation of FIG. 1B, a left-click upon thehighlighted telephone number automatically initiates an operation, suchas calling the telephone number, and a right-click makes a popup menuappear, with various options of what action to take with or upon thetelephone number. Thus, if the user made a left-click in step 161, theselected telephone number is extracted from the computer document andforwarded to another program or device. Although the presently preferredembodiments send the extracted telephone number to a program forperforming IP telephony, it should be understood that the extractedtelephone number may be forwarded to any kind of software (such as anaddress book program or the clipboard) or hardware (such as an actualtelephone connected to the computer, or an output port for transmissionto any other sort of device). If the target is a program, the targetprogram may also need to be initiated or loaded in this step.

If the user has made a right-click in step 161, a menu of options isdisplayed to the user in step 163. This menu of options may includeautomatically calling the telephone number with the program (which maybe equivalent to step 162). Other options include changing the telephonenumber (e.g., adding a prefix either for an international call or forgetting a direct line out of an office) and then auto-dialing it, savingthe telephone number to a specific place (e.g., an address book), savingit to the computer clipboard, etc. The selected action (or no action atall, if no action was selected) is performed in step 164.

In a preferred embodiment of the inventive method, the user is given theoption of storing the telephone number with or without contextinformation. If the user elects to store the context information, thetelephone number is also stored. Specifically, the selected informationis identified as a telephone number and is sent from the node in whichit was found to a desired location, in a desired form, such as in theform of a contact file in a contact manager program. In this example, anew contact file is created and the telephone number is automaticallyplaced into the correct field in the contact file so that the contactmanager program will recognize the file as a new contact having theselected telephone number. Context information is then also gatheredfrom the information structure of the computer document. For example,the data contained in nodes within a predetermined proximity to the nodecontaining the telephone number may be recognized as providing thecontext information.

The above example is illustrated by FIG. 1D. In one embodiment of theinvention, the computer document being parsed is a web page and, forthis example, it is presumed that the information contained in the webpage has been sorted into a virtual tree, such as shown at 20 in FIG.1D. It is also presumed in this example that the desired telephonenumber is found at node 34 and the web page may be found at a specificURL which would be root node 22. The web page may be the home page for aparticular company whose identity is found at node 24. Contact persons(e.g., a sales representative 26, or technical support contact 28) maybe identified in the web page giving the user the opportunity to contactthe identified individuals. In the case of the technical support contact28, the contact information may include the hours during which technicalsupport is open (contained in node 30), a telephone number 34 and ane-mail address 36. Presuming that the user does not wish to contacttechnical support personnel immediately, but may be interested incontacting them at a later date without having to find the web pagecontaining the telephone number again, it would be useful to permit theuser to save the located telephone number and the context information,such as the hours at which the technical support number may be called,the name of a particular individual, and the like. This would then, inthis example, be considered “context information”, i.e., informationfound in the computer document in which the telephone number is presentthat serves to give added meaning or “context” to the telephone number.The context information may be identified by its “physical” proximity tothe telephone number in the context of the hierarchical tree or by itshierarchical relationship to the telephone number, depending upon theapplication.

In this example, the context information for the user would comprise theinformation found within a predetermined number of nodes of thetelephone number, either “up” or “down” the tree. This proximity couldbe varied by the user to gather more or less information as desired, andmay be varied, depending upon the application, “on the fly”, i.e., whilethe context information is being gathered.

In other examples, it may be desired to save context information notbased upon its relative proximity to the telephone number, but ratherbased upon its “absolute” position in the hierarchy, such as the “root”URL 22 of the web page, no matter how “far” node 34 containing thetelephone number may be from the root 22 in tree 20. This may also bedone, depending upon the dictates of the application, and the person ofordinary skill may make that determination without undueexperimentation.

Presuming the user determines that the context information identified issatisfactory, the user may then direct the telephone number to be savedtogether with the context information. The desired telephone number maybe stored as a contact file in an existing contacts manager program(e.g. as Outlook), and would therefore be saved in the form and locationof a file suitable for use by the user's contact manager. The contextinformation may be recognized as filling certain standard fields of sucha file, such as name, address, or the like, and would therefore bestored in a file in the appropriate field thereof so as to be recognizedby the program as information of the relevant type. The user may also begiven the option of not storing the context information, or of storingit in a different location.

If the context information is not stored automatically in a file of apredetermined type, the context information may be stored at a locationrelated to the location in which the telephone number is stored, such asin a location at a predetermined relationship to the telephone number(e.g., as adjacent thereto) or at another location linked to thetelephone number by a pointer.

In yet another embodiment, the user may be given the option of addingcontext information manually, for example where the “context” is notfound in the computer document itself but is added by the user (e.g.,where the technical support number being stored relates to “my son'scomputer” or “my office computer”, to differentiate it from othertechnical support numbers which may also be stored).

In any event, the inventive method permits this context information tobe stored with the telephone number, in a manner which enables the userto recall the telephone number and the context information.

According to still another embodiment of the invention, if the user isgiven the option of establishing a telephone connection directly fromthe application, the connection may be established via the user'scomputer, such as through a computer-to-computer connection. One type ofsuch connection is through a voice over internet protocol, (VOIP) forexample the Session Initiation Protocol (SIP).

If the telephone call is established via a connection directly from theuser's computer, the context information may be transmitted to thereceiving device, for example the screen of a computer which correspondsto the number being dialed, or a database associated with the telephonenumber being dialed. In this instance, the context information mayinform the called computer of the context in which the caller discoveredthe telephone number. This may be useful if the telephone number wasfound in an advertisement, so that the receiving party may track theeffectiveness of various advertisements in promoting active sales leads.The context information may also be sent to any other kind of receivingdevice, such as a fax machine, or to a telephone which receives thephone call (such as through the use of the “Caller-ID” feature of thereceiving telephone), so that the receiving telephone may identify thecaller as having been referred to the receiving telephone number from aspecific source.

Transmitted context information may also be used to transmit the type ofdocument in which the telephone number was found, such as a web pageadvertisement, a word processing document, a spreadsheet or an e-mail.The context information in this latter case may include the sender ofthe e-mail, the recipient(s) thereof or the “subject” line of thee-mail, so that the receiving party may use the context information todetermine how best to respond to the telephone call, or to evaluate therelative effectiveness of different types of solicitations of telephonecalls.

Returning to FIG. 1B, after either the appropriate left-click orright-click action is performed in step 162 or step 164, respectively,the process returns to monitoring in step 151.

As mentioned above, FIG. 1C is a general schematic flowchart like FIG.1A, but FIG. 1C has additional steps for embodiments in which theintegrity of the computer document is preserved in order that theoriginal format of the computer document may be used in any file actions(e.g., saving, forwarding, copying, or any other operation where theoriginal format of the computer document is needed or desired). In theseembodiments, a provisional computer document is used in order tovisually render the computer document for the user, so that the tags forhighlighting and functionality are placed into this provisional computerdocument rather than the original computer document. In FIG. 1C, theoriginal computer document is modified to create the provisional viewingcomputer document, and then the original computer document is recreatedfrom the provisional viewing document, but other embodiments may storethe original computer document and destroy the provisional viewingcomputer document when it is no longer being presented to the user.Furthermore, the provisional viewing computer document may be storedpermanently so that either the process does not need to be repeated eachtime the user views the computer document, or the provisional computerdocument can be used to store additional information, such as if a callwas made to one of the highlighted telephone numbers, and when. In oneembodiment, there may be a value added service which performs the stepsand stores the provisional viewing computer documents for use by userclients.

In FIG. 1C, steps 100-110-115-120 are performed as described inreference to FIG. 1A. In step 170 of FIG. 1C, the provisional viewingdocument is created from the original computer document. As will bedescribed in reference to the third preferred embodiment below, theoriginal computer document may be a plaintext e-mail, while theprovisional viewing computer document may be a markup language document(so that it may be used to render the highlighted telephone numbers inthe e-mail GUI, and provide the requisite functionality).

Although step 170 appears after step 120 in FIG. 1C, the step ofcreating the provisional viewing computer document may be performed atan earlier time during the method (such as simultaneously with, orimmediately after, step 100). Furthermore, in the third presentlypreferred embodiment of the invention, step 170 may only occur if one ormore instances of telephone numbers are found on the computer document(or on the provisional viewing computer document).

In those embodiments in which the original computer document ispreserved, the context information may include information which ispresent in the original computer document but not in the provisionaldocument, such as stripped metatags or “salt” tags which are notsearched for the actual telephone numbers. In such an embodiment, thedefinition of the proximity of the context information would indicatethat the context information is found in the metatag in the originalcomputer document. This may arise where the desired context informationis, for example, information which is not displayed in the computerdocument, but is embedded therein.

In step 133 of FIG. 1C, tags are placed around each telephone numberidentified in step 120 in the provisional viewing document. Theprovisional viewing computer document is used to visually render thecomputer document in step 143, whereby the identified telephone numbersare highlighted in the GUI given additional capabilities. In step 150,the method awaits an indication that the user has interacted with ahighlighted telephone number in the visually rendered provisionalviewing computer document. If an event is detected which indicates thatthe user has interacted with a highlighted telephone number,specifically, if the user has indicated a desired operation, the desiredoperation indicated by the user is performed in step 160.

However, if it is indicated in step 180 that the user (or the system,etc.) is going to perform a file action (e.g., saving the computerdocument), the original computer document is recreated from theprovisional viewing computer document in step 190. The file action isthen performed on the recreated original computer document in step 195.As stated above, in other embodiments, the original computer documentmay never be destroyed (as it is when modified in step 170 to become theprovisional viewing computer document), but rather stored while theprovisional viewing computer document is being used to render it.Furthermore, in embodiments where the original document is modified instep 170 to create the provisional viewing computer document, theoriginal computer document is recreated in step 190 without affectingthe provisional viewing computer document (i.e., the provisional viewingcomputer document remains intact—is still being used to visually renderthe document in the GUI).

Three presently preferred embodiments are described below: a “web-dial”embodiment in which the computer document is a markup language document(specifically, an HTML document) is described with reference to FIGS.2A-2B; an “office-dial” embodiment in which the computer document is anywhich can be used in a suite of productivity applications (specifically,files used by a productivity suite including a word processing program,a spreadsheet program, and a slide/visual presentation program) isdescribed with reference to FIGS. 3A-3B; and an “e-mail-dial” embodimentin which the computer document is a transmitted message (specifically,an e-mail) is described with reference to FIGS. 4A-4B.

Although the screen shots of FIGS. 2A, 3A, and 4A are from the monitorof a desktop personal computer, the present invention may be implementedon any type of computing device, including, for example, laptopcomputers, Personal Digital Assistants (PDAs), telephones,appropriately-equipped television receivers, or any otherappropriately-equipped appliance.

Web-Dial

In the first presently preferred embodiment of the present invention, asshown in FIGS. 2A-2B, the computer document is a markup languagedocument, such as a web page, which is being opened in a browser. Thefirst presently preferred embodiment is similar to the inventiondescribed in the aforementioned U.S. patent application Ser. No.10/697,468 (the '468 application), which was also invented by thepresent inventor, and from which the present application claimspriority. That application has been incorporated by reference, and maybe referred to for guidance on some particular details of the firstpresently preferred embodiment herein; however, there are differencesbetween the implementation of this embodiment and the implementation inthe '468 application, as will be apparent to one skilled in the art whenreading the following descriptions.

In FIGS. 2A-2B, the markup language document is an HTML (HyperTextMarkup Language) document, but the first presently preferred embodimentmay be applied to any type of markup language document. The “hypertext”in HTML refers to the content of web pages—more than mere text,hypertext (sometimes referred to as “hypermedia”) informs the webbrowser how to rebuild the web page, and provides for hyperlinks toother web pages, as well as pointers to other resources. HTML is a“markup” language because it describes how documents are to beformatted. Although all web pages are written in a version of HTML (orother similar markup languages), the user never sees the HTML, but onlythe results of the HTML instructions. For example, the HTML in a webpage may instruct the web browser to retrieve a particular photographstored at a particular location, and show the photograph in the lowerleft-hand corner of the web page. The user, on the other hand, only seesthe photograph in the lower left-hand corner. HTML is also a variant ofXML (eXtensible Markup Language). The difference between the two markuplanguages is that HTML was designed to display data and focus on howdata looks, whereas XML was designed to describe data and focus on whatdata is. XML is a universal storage format for any type of data andfiles make them portable between different hardware, software, andprogramming languages.

In FIG. 2A, a screen shot of a GUI window 200 of a standard web browser(in this case, Internet Explorer) is shown with a loaded web page 210.In FIG. 2A, there is a ‘bookmarklet’ 240 labeled “IP-Phone” in the Linkstoolbar at the top of the web browser GUI; this bookmarklet 240 can beused to turn the Web-Dial functionality on or off. When the Web-Dialfunctionality is turned off, the telephone numbers are not highlightedor indicated in any way (i.e., the web page appears as it would appearin a standard, prior art web browser). Because the Web-Dialfunctionality has been turned on in FIG. 2A, web page 210 has beenparsed and classified according to steps 110 and 120 in FIG. 1A and thetelephone numbers that were identified in step 120 are highlighted inFIG. 2A according to step 140 in FIG. 1A. Specifically, three telephonenumbers 211, 212, and 213 can be seen highlighted on web page 210.

In the first presently preferred embodiment, when the cursor is locatedover a highlighted telephone number, it changes from its usual arrowshape to an icon indicating that particular functionality is availablefor the highlighted telephone number if the user right- or left-clicks.In FIG. 2A, the user has right-clicked highlighted telephone number 211,thereby causing menu 230 to pop up, in accordance with step 163 in FIG.1B. If the user had left-clicked, highlighted telephone number 211 wouldhave been extracted and sent to an IP telephony program (i.e. step 162in FIG. 1B).

Although menu 230 is shown immediately above and to the right ofhighlighted telephone number 211 in FIG. 2A, menu 230 may appear atother positions on the computer screen. The options presented to theuser in pop up menu 230 of FIG. 2A are “Direct Dial”, “(+1) DialExternal”, “(+1) Dial Out State”, “Disconnect”, “Save to address book”,“Copy number”, “Copy dial format”, and “Information”. In FIG. 2A, theuser has moved the cursor to select the “Save to address book” option inpop up menu 230. The first three options in pop up menu 210 areself-explanatory.

“Disconnect” lets the user disconnect the phone line after a telephonecall. This function would be grayed-out when the user initially selectsthe highlighted telephone number, as there would be no connection yet.Of course, it is also possible that after finishing a call using the IPtelephony program, the user may disconnect using the IP telephonyprogram. “Copy number” copies the number into the computer's clipboardfor later use by the user. “Copy dial format” indicates that the formatof the highlighted telephone number is a format that should be searchedfor in the future when step 220 is performed. “Information” providesinformation about the web-dial functionality. The “Copy Dial Format”provides the ability to learn. In the presently preferred embodiments,feedback received from the user can be used to modify or add to thesearch method used to detect any instance of a telephone number.

It should be understood that the user options shown in the pop-up menuof this exemplary GUI are exemplary, and that many variations regardingthe options are possible: there may be greater or fewer options, ordifferent options, such as reverse telephone number lookup, which wouldaccess a reverse phone lookup directory to verify or to obtain theaddress and name associated with the highlighted telephone number.

According to all the presently preferred embodiments, the user maymanually select a telephone number and pass the telephone number to theIP telephony program (or perform other operations upon or with thetelephone number, such as store it in an address book). To do so in thefirst presently preferred embodiment, the user would select the numberson the web page using the cursor, and then click bookmarklet 360, whichcauses the telephone number to be sent to the IP telephony program. Thismanual selection is provided for those cases where the telephone numbersare not identified on the web page (and thus are not clickable), or incase a telephone number has been saved in temporary memory (such as thecomputer clipboard) and now the user wishes to dial it, or when the userhas turned off the Web-Dial functionality.

FIG. 2B is a schematic diagram of components performing the functionsresulting in the capability and GUI functionality shown in FIG. 2A. InFIG. 2B, a web browser 250, such as Internet Explorer, receives webdocuments and user input or interaction. WebDial Plugin 260 is a BrowserHelper Object (BHO) that is loaded and initialized whenever a newinstance of web browser 250 is started. BHOs, such as WebDial Plugin260, are Dynamic Link Libraries (DLLs) which run in the same memorycontext as web browser 250 and can perform any action on the availablewindows and modules. The WebDial Plugin 260 is typically unloaded whenthe instance of web browser 250 is destroyed, although this may not betrue in some embodiments.

Event Handler 262 in WebDial Plugin 260 listens for events occurring inweb browser 250. Specifically, the Event Handler 262 listens for twoevents that are fired by web browser 250 at specific times: theDocumentComplete event which is fired when a web document, with all ofits forms, has fully loaded into the browser; and the BeforeNavigateevent which is fired when the user has performed an action which seemsto require that a new web document be loaded into the web browser. Suchuser actions include clicking the Back or Forward buttons, or entering anew URL into the navigation bar.

If a DocumentComplete event is detected, the remaining modules inWebDial Plugin 260 go to work. Specifically, Parser 264 parses therun-time Document Object Model (DOM) of the downloaded document throughthe DOM application programming interface (API) 252. The Document ObjectModel (DOM) is a platform- and language-neutral API standard that allowsprograms and scripts to dynamically access and update the content,structure, and style of documents (both HTML and XML). Using DOM API252, a document can be further processed and the results of thatprocessing can be incorporated back into the page presented to the userby the renderer 254. In essence, the DOM API provides a tree-like model,or framework, of the objects in a document, i.e., when an XML/HTMLdocument is loaded into an application (such as a web browser likeInternet Explorer), the DOM API creates a DOM of the downloaded documentin the form of an in-memory tree representation of the objects in thatdocument. Using the DOM API, the run-time DOM may be used to access,traverse (i.e., search for particular objects), and change the contentof the downloaded document.

The DOM of the downloaded web page in the first preferred embodiment mayoperate as the provisional viewing computer document discussed inreference to FIG. 1C above.

By contrast with the embodiment described in the '468 application, theWebDial Plugin 260 of the first presently preferred embodiment hereindoes not insert Javascript code into the DOM API 252 of Web Browser 250.Instead, modules within WebDial Plugin 260 perform most of the work.Naturally, these modules are written in C++ as part of a DLL rather thanJavascript code to be inserted in the DOM, as in the embodimentsdescribed in the '468 application.

Parser 264 parses the DOM of the downloaded document to find anyinstances of telephone numbers. Specifically, parser 264 uses regularexpression parsing rules designed to pick up the various formats thatphone numbers can take within a computer document. An exemplary parsingimplementation is described below; however, it should be noted that thisparsing implementation is only one of a myriad of possible searchexpressions when implementing the present invention.

The following characteristics were used to define North American basedtelephone numbers:

-   -   a) The area code is 3 digits and can sometimes be in brackets;    -   b) The subscriber number is seven digits long and is clustered        into two groups comprised of a first group of three digits and a        second group of four digits;    -   c) The two groups are separated by one of a space, dash, dot, or        a middle dot, or are not separated at all;    -   d) The area code and the subscriber number are separated by one        of a space, dash, dot, or a middle dot, or not separated at all;    -   e) The phone numbers may begin with a + character followed by        the digit 1, the + character followed by a space and then the        digit 1; and    -   f) The optional ‘+1’ and ‘+1’ in (e) can be separated from the        10 digit phone numbers with one of a space, a dash or a dot.

An example of a regular expression string which satisfies the abovecharacteristics is:

-   -   Λ+? ?1? ?−?\.?\(?\d{3}\)?\·?−?\.? ?\d{3}−?.?\·? ?\d{4}/gi

The regular expression string is bounded by the “1” character. Next the\+ matches the + character. The plus has a “\” in front to specify wemean an actual “+” character and not the special character. This isfollowed by the “?” character which means that it matches zero or oneinstance of the “+” character. This is followed by the space and “?”character which matches zero or one instance of a space character. Therest of the regular expression captures the characteristics that wedesire to capture. At the end the “g” character indicates that a globalmatch be performed to get all instances of such pattern and the “i”character makes the match case insensitive.

The international numbers can be quite varied in format and it wasdiscovered to be best if the parsing was limited to the characteristicsdescribed below:

-   -   a) The number begins with a “+” character, followed by 1 to 3        digits which signifies the country code, then followed by an        optional space character;    -   b) A 1 to 5 digit string comprises the area code and can        sometimes be in brackets; and    -   c) The 1-5 digit area code is followed by four groups of digits,        in which the groups are separated by a zero or one instance of a        space character.

An example of a regular expression string which satisfies the abovecharacteristics is:

-   -   /(\+\d{1,3} ?)(\(\d{1,5}{cube root})|\d{1,5}) ?\d{1,6} ?\d{0,7}        ?\d{0,5} ?\d{0,5}/gi;

With these two examples, it should be clear how such a string can bewritten and how the parsing can be performed. The same principles may beused in a similar manner to implement parsing in order to detect othertypes of data on any type of computer document.

Once parser 264 has completed its task, the results, if any, are outputto classifier 266. Classifier 266 extracts lexicalized contextualfeatures for each result, i.e., for each possible telephone number inthe computer document, it extracts local features from the computerdocument to determine the likelihood that the parser-selected number isindeed a telephone number. For example, if the possible telephone numberis preceded by certain keywords, such as “telephone”, “phone”, “mobile”,“cell”, and “fax”, it strongly indicates that the current number is infact a phone number. As another example, icons, such as a telephoneimage, may also increase the likelihood that the number is a telephonenumber. Additional search parameters are possible, such as theformatting of the page around the number (e.g., if it appears to be partof a contact block on that webpage), and other terms and icons, whensearching for instances of telephone numbers, as would be known to oneof ordinary skill in the art.

The classifier 266 can be pre-trained (i.e., before installation) byperforming statistical analyses of web pages, and specifically thetypical format of, and the contextual features around, telephone numbersfound therein. It's also possible that the classifier 266 can be trainedby the particular user of the WebDial Plugin 260, by having a webinterface for the user to identify telephone numbers (and telephonenumber formats and contexts), and thereby train, the classifier 266 inWebDial Plugin 260. The present embodiments of the invention use aSupport Vector Machine as classifier 266, but any conventionalclassifier can be used, as would be known to one of ordinary skill inthe art. Even without an entire user training web interface, classifier266 can receive feedback from the user and then use that feedback inorder to modify or add to the search method used to detect any instanceof a telephone number. Furthermore, it is contemplated that searchparameters may be under the control of the user, e.g., the user may havethe ability to turn on and off certain search parameters, in addition tothe user defining words, terms, formats, or contexts to look for in theweb page.

Having analyzed the output of parser 264 to determine which of theparsed telephone numbers is most likely to be a telephone number,classifier 266 then outputs the most likely numbers to be telephonenumbers to Markup Module 268, which ensures that these most likelytelephone numbers in the downloaded document are highlighted and madeclickable.

Markup Module 268 creates a span tag which is placed around thetelephone number in the downloaded document (or, more exactly, the DOMof the downloaded web page). The span tag is used in HTML to groupinline-elements together in order to apply specific style and/or eventattributes to the elements within the “span”. In this case, the span tagindicates that the grouped elements, i.e., the numbers making up thephone number, will be highlighted and active upon certain cursor events.Specifically, when the cursor is over the highlighted area, the cursorturns into a telephone icon; when the highlighted phone number isleft-clicked, it is interpreted as a hyperlink click which isintercepted by Event Handler 262 and results in the IP telephony program270 being called; and, when the highlighted phone number isright-clicked, the menu 230 of options in FIG. 2A pops up.

If a click is made over one of the phone number span tags, a new type ofURL is captured by the BeforeNavigate event. In accordance with thespecific implementation of the presently preferred embodiments, theinventive format of this new “phone” URL comprises “phone://<strippedtelephone number>”. In other words, the mouse click is interpreted bythe web browser as a navigation event, i.e. a call to load another webpage, and thus the web browser begins the process of navigating to thepage indicated by the mouse click, but instead of a conventional URL,the new inventive phone URL is captured by the BeforeNavigate event,which, in turn, is intercepted by Event Handler 262.

If a BeforeNavigate event is heard by Event Handler 262, WebDial Plugin260 intercepts the URL to which the web browser is navigating todetermine whether it is a phone URL generated by clicking on one of thehighlighted telephone numbers on the webpage. If Event Handler 262determines that it is a phone URL, WebDial Plugin 260 extracts thenumber string from the phone URL and sends the number string to IPTelephony program 270. The number string which makes up the highlightedtelephone number may be stripped of any separator characters, such asspaces, parentheses, brackets, etc., before passing it on; however, thisstripping routine is not necessary, and the presently preferredembodiments do not have such a stripping routine because the IPtelephony program 270 may use some of these separator characters toproperly interpret the telephone number it represents.

Thus, it can be seen that the specific implementation of the presentlypreferred embodiment uses the normal functions and events (e.g., theDocumentComplete and BeforeNavigate events, the BHOs and DLLfunctionality, etc.) of the web browser in a new, inventive manner inorder to provide some of the features of the present invention.

Although FIGS. 2A-2B show an Internet Explorer/Windows implementation ofthe first presently preferred embodiment, all of the presently preferredembodiments of the present invention is by no means limited to eitherthe Microsoft Windows operating system or the Internet Explorer webbrowser. Other web and/or non-web browsers, such as Netscape Navigator,Apple's Safari, Mozilla, Opera, etc., may be used with all of thepresently preferred embodiments.

Furthermore, although all three presently preferred embodiments aredescribed in the context of either the Microsoft Windows operatingsystem or one of the Microsoft software applications, it is contemplatedthat the present invention (and any of the three presently preferredembodiments) may be implemented in a system running any operatingsystem, such as the Apple Mac OS, the Linux operating system, or any ofthe flavors of UNIX offered by the larger computer system designers(e.g., Solaris on Sun computer systems; Irix from Silicon Graphics,etc.). In other words, the present invention is platform-independent.

The first presently preferred embodiment has been described in fairlygreat detail above, and it should be understood that many of thesespecific details are also applicable in the other presently preferredembodiments, and thus, do not need to be described again. As oneexample, the regular expressions used by Parser 264 may also be used inthe two embodiments described below. As another example, the optionsdescribed in reference to the right-click pop-up menu would also beavailable in the two embodiments described below. In short, most of thefeatures and functionality described above are equally applicable to thetwo embodiments described below, as would be understood by one skilledin the art.

Office-Dial

In the second presently preferred embodiment of the present invention,as shown in FIGS. 3A-3B, the computer document is an office productivitysuite document, specifically, a word processing document. The exemplaryword processing document in FIG. 3A is an MS Word document, although thefunctionality described in the second presently preferred embodiment maybe used with any of the MS Office documents (e.g., MS Excel, MSPowerPoint, etc.). Furthermore, the second presently preferredembodiment may be implemented in any office productivity application,such as a word processing application, a spreadsheet application, aslide presentation application, a calendar application, etc.

In FIG. 3A, a screen shot of a GUI window 300 of a word processingprogram (in this case, MS Word) is shown with a loaded word processingdocument 310. Word processing document 310 has been parsed andclassified according to steps 110 and 120 in FIG. 1A and the telephonenumbers that were detected in step 120 are underlined in FIG. 3Aaccording to step 140 in FIG. 1A. In the second presently preferredembodiment, when the cursor is located over a underlined telephonenumber (e.g., phone number 311), an icon labeled “i” (e.g., box 320)appears above the underlined telephone number indicating that particularfunctionality is available for the underlined telephone number. The “i”icon, and other features of the second presently preferred embodiment,are provided by the Smart Tags technology, as will be described ingreater detail below. In FIG. 3A, the user has clicked the “i” icon 320,thereby causing menu 330 to appear, similar to the right-click option ofstep 163 in FIG. 1B.

Although the “i” icon 320 is shown immediately above and to the right ofunderlined telephone number 311 in FIG. 3A, “i” icon 320 may appear atother positions on the computer screen. The options presented to theuser in pop down menu 330 of FIG. 3A are “Call Number”, “Remove thisSmart Tag”, “Stop Recognizing “1-877-372-5719”, and “Smart Tag Options”.In FIG. 3A, the user has selected the “Call Number” option in pop downmenu 330, thereby causing the Call dialog box 374 to appear above the MSWord GUI 300. The user has the option buttons “call”, “cancel”, and/or“help” in the Call dialog box 374. In addition, the Call dialog box 374has the check box option to not show this message again” so that thenext time the user selects “Call Number”, the Avaya IP Softphone 376 isautomatically opened and dialing commenced. In FIG. 3A, the user hasselected the “Call” button in Call Dialog box 374 so the Avaya IPSoftphone 376 has opened and is calling the selected number. The “StopRecognizing “1-877-372-5719” ” option in pop down menu 330 allows theunderlining of telephone number 311 to disappear, as well as thetelephone functionality for telephone number 311. The “Smart TagOptions” and “Remove the Smart Tag” options in pop down menu 330 will beunderstood when the Smart Tag architecture and functionality isdescribed further below.

Other options could appear in pop down menu 330 in other implementationsof the second presently preferred embodiment, as would be known to oneof ordinary skill in the art, such as, inter alia, the options shown inpop up menu 230 in FIG. 2A.

Although not shown in FIG. 3A, a IP-phone bookmarklet may appear inother embodiments in the standard toolbar at the top of the MS Word GUI300; this bookmarklet could be used to turn the Office-Dialfunctionality on or off. When the Office-Dial functionality is turnedoff, the telephone numbers are not underlined or indicated in any way(i.e., the word processing document appears as it would appear in astandard, prior art word processing GUI).

This implementation of the second presently preferred embodiment usesSmart Tag functionality and architecture, which will be described belowwith reference to FIG. 3B, to automatically search for telephone numbersin any MS Office file (including MS Word, MS Excel, MS Power Point,etc.) being presently viewed by the user, and then underlining any foundtelephone numbers, as well as providing the menu and optionfunctionality, an example of which is shown in FIG. 3A. However, theparser and classifier aspects of the second presently preferredembodiment are novel, as is the general idea embodied therein.

FIG. 3B illustrates the architecture of the Smart Tags implementation ofthe second presently preferred embodiment in an MS Office environment.In FIG. 3B, an MS Office program 350 (in this instance, MS Word) is incommunication with a Smart Tags DLL (in this instance, Office-Dial 360)which is comprised of STRecognizer 365 and STAction 367, and STAction367, in turn, is in communication with Avaya IP SoftPhone 370. As shownby the arrows with boxed numbers in FIG. 3B, when a computer document,such as a word processing document, is downloaded into an MS Officeprogram 350, such as MS Word, STRecognizer 365 in Office-Dial Plugin 360parses the content of the downloaded computer document to find anyinstances of telephone numbers (step 1). STRecognizer 365 uses theregular search expressions in the RegExp module to search for thetelephone numbers. The research string expressions would be similar tothe ones described in reference to FIG. 2B above, as would be understoodby one of ordinary skill in the art.

If any telephone numbers have been found, STRecognizer 365 applies SmartTags to each found instance of a telephone number (step 2). Theseinserted Smart Tags provide the highlighting (i.e., underlining) of eachtelephone number, as well as the capabilities and GUI functionalityshown in FIG. 3A. Referring back to FIG. 3A, as well as FIG. 3B, if theuser puts the cursor over an underlined telephone number 311, the “i”button 320 will appear in the GUI window above the underlined telephonenumber, and STAction 367 will be called in order to show, and to providethe functionality for, the menu of commands 330 (step 3). If the userselects the “Call Number” button on menu 330, STAction 367 will call andprovide the appropriate number string to Avaya IPSoftphone 370, whichwill perform the dialing and connecting (step 4).

Smart Tag technology was first introduced in Microsoft Office XP toenable users to dynamically present additional information in theircomputer documents. For general users of MS Office, Smart Tags areintended to boost productivity by having actions linked to keywords.Smart Tags may be used to link users to internal resources based on thetype of data either in or input into computer documents in MS Officeapplications. For example, a purchase order number in an Excelspreadsheet may be linked to detailed information about the sale from asource on a company intranet.

Smart Tags are enabled by two simple DLLs: a recognizer module (“SmartTag Recognizer” or “STRecognizer”) and an action handler (“Smart TagAction” module or “STAction”). When a Smart Tag-enabled application isstarted up, it fires up all available Smart Tag recognizer modules andaction handlers. When a user either opens a computer document orprovides input into a computer document, the recognizer module parsesthe content of the opening computer document and/or the input of theuser to determine if any of the specified data type to be recognized isthere. When the data type is recognized, the recognizer module insertsSmart Tags around the identified datatype, which will underline theidentified data type and provide the “i” button next to the identifieddata type. In the second presently preferred embodiment, theSTRecognizer merely identifies possible telephone numbers, while theparser and classifier functions are implemented as described herein. Theaction handler provides the menu options shown when the user clicks the“i” button, and provides the functionality or calls other programs toprovide said functionality.

Smart Tags are similar to markup language tags and, in fact, the SmartTags technology uses an XML namespace to enable the functionality. Thus,like markup language tags, Smart Tags are hidden inside the computerdocument. If they could be seen, they would appear like any other pairof XML tags surrounding the text, like: <st1:inventor>RamKashi</st1:inventor>. For more information concerning Smart Tagsarchitecture and technology, please see A Developer's Take on SmartTags, Bill Coan, February 2005; Developing Simple Smart Tags, PaulCornell, May 2001; Building Smart Tags in Microsoft Visual Basic .NET,J. Sawyer, October 2001; Developing Smart Tag DLLs, Paul Cornell, April2001; and Regular Expression Support in Microsoft Office System SmartTags, Markus Egger, August 2003. All of these articles may be foundonline at the Microsoft Developers Network (MSDN) Library atmsdn.microsoft.com, and all of these articles are hereby incorporated byreference in their entirety.

E-Mail-Dial

In the third presently preferred embodiment of the present invention, asshown in FIGS. 4A-4B, the computer document is a message, specifically,an e-mail which is being viewed in MS Outlook, although thefunctionality described in the third presently preferred embodiment maybe used with other e-mail applications, such as Apple Mail, Eudora,Mozilla, Firefox, etc. In the third presently preferred embodiment,telephone numbers are automatically detected and highlighted, regardlessof whether the format of the original message is Plain Text (Plain),Rich Text Format (RTF), or HTML. In order to highlight telephone numbersin any email regardless of format, the third presently preferredembodiment creates a provisional viewing e-mail at runtime in which tagscan be placed so that detected telephone numbers are appropriatelyhighlighted, and have the appropriate functionality, similar to themethod of FIG. 1C. After viewing, and/or before any action is performedon the original e-mail, any placed tags are removed from the provisionalviewing e-mail.

In FIG. 4A, the two GUI windows used to organize and present informationin the e-mail program MS Outlook are shown: the Explorer GUI window 400and the Inspector GUI window 401. Explorer window 400 is the main GUIfor the MS Outlook program, and it gives a user access to each of thefolders that make up Outlook: Inbox, Task, Contacts, Calendar, Journals,and Notes folders. The display can be customized on a folder-by-folderbasis, such as by specifying which columns are shown in Explorer window400. Explorer window 400 has several panes: the Outlook bar 491, whichin FIG. 4A is showing the Mail folder, on the left-hand side of Explorerwindow 400; the view pane 493 which in FIG. 4A is showing the list ofe-mails in the Inbox on the right-hand top of Explorer window 400; andpreview pane 495 which in FIG. 4A is showing the e-mail presentlyselected in view pane 493 on the right-hand bottom of Explorer window400. The use and meaning of both Explorer window 400 and Inspectorwindow 401, as well as any panes therein, would be understood by one ofordinary skill in the art.

Inspector window 401 is used to display one item at a time: when a userselects an item in Explorer window 400, a new Inspector window 401 willappear, showing the selected item. In FIG. 4A, the user hasdouble-clicked on an e-mail 411 shown in either the view pane 493 or thepreview pane 495 of Explorer window 400, and thus a separate Inspectorwindow 401 appears showing the details of the selected e-mail 411.

In e-mail 410, telephone number 411 is shown underlined in both previewpane 495 of Explorer window 400 and Inspector window 401. Thus, e-mail410 has been parsed and classified according to steps 110 and 120 inFIG. 1A and the telephone numbers that were detected in step 120 areunderlined in FIG. 4A according to step 140 in FIG. 1A. Similar to thesecond presently preferred embodiment, in the third presently preferredembodiment, when the cursor is placed over a underlined telephone number(e.g., phone number 411), an “i” icon box 420 appears above theunderlined telephone number indicating that particular functionality isavailable for the underlined telephone number. In the Inspector window401 of FIG. 4A, the user has clicked the “i” icon 420, thereby causingpull-down menu 430 to appear, similar to step 163 in FIG. 1B.

Although the “i” icon 420 is shown immediately above and to the right ofunderlined telephone number 411 in FIG. 4A, “i” icon 420 may appear atother positions on the computer screen. Furthermore, although when thecursor is located over the span of a underlined telephone number, the‘i’ icon 420 of a Smart Tag appears in the second and third presentlypreferred embodiments, it should be understood that any icon and/orvisual representation could be used to indicate the availablefunctionality for the underlined telephone number. For example, the iconcould be the image of a telephone. The icon could also be permanentlyrepresented, i.e., appear at all times next to the highlighted telephonenumber, even when the cursor is nowhere near the highlighted telephonenumber.

The only option presented to the user in pop down menu 430 of FIG. 4A is“Call Number”. Other options could appear in pop down menu 430 in otherimplementations of the third presently preferred embodiment, as would beknown to one of ordinary skill in the art, such as, inter alia, theoptions shown in pop up menu 230 in FIG. 2A and pop down menu 330 inFIG. 3A, or different options, such as reverse telephone number lookup,which would access a reverse phone lookup directory to verify or toobtain the address and name associated with the highlighted telephonenumber.

Although not shown in FIG. 4A, an IP-phone bookmarklet could appear inother embodiments in the standard toolbar at the top of Explorer window400 and/or Inspector window 401; this bookmarklet could be used to turnthe Email-Dial functionality on or off. When the Email-Dialfunctionality is turned off, the telephone numbers are not underlined orindicated in any way (i.e., the e-mail appears as it would appear in astandard, prior art e-mail GUI). Furthermore, in some embodiments havinga bookmarklet, the user may use the cursor to select a telephone numberin an email and then press the bookmarklet button to pass the telephonenumber to an IP telephony program or to perform other operations upon orwith the telephone number, such as store it in an address book. Thismanual selection capability could also be provided for those cases wherethe telephone numbers are not identified or underlined on a documentpage (and thus are not clickable), or when a telephone number has beensaved in provisional memory (such as the computer clipboard) and now theuser wishes to dial it.

According to any of the presently preferred embodiments, thefunctionality provided for the highlighted telephone number may vary.For example, one left-click could result in the telephone number beingautomatically dialed (without any further interaction by the user). Asanother example, left- and right-clicks may show different pop-downmenus for different purposes.

FIG. 4B is a schematic diagram of components performing the functionsresulting in the capability and GUI functionality shown in FIG. 4A. InFIG. 4B, an email program, in this case, MS Outlook 450, receives emailsin any of Plain, RTF, or HTML format as well as user input orinteraction. Email-Dial Add-in 460 is a COM (Component Object Model)Add-in, a DLL, which is registered with MS Outlook 450 so that it isloaded and initialized whenever a new instance of MS Outlook 450 isstarted. Thus, Email-Dial Add-in 460 loads into MS Outlook 460's addressspace and can perform any action on the available windows and modules inMS Outlook 460. Email-Dial Add-in 460 is typically unloaded when theinstance of MS Outlook 450 is destroyed, although this may not be truein some embodiments.

An “object” in object-oriented programming (and thus in any ObjectModel, including Outlook Object Model 462) is generally a self-containedentity that consists of both data and procedures to manipulate the data.Thus, almost anything in the programming environment is an object, andan Object Model, such as Outlook Object Model 462, allows one to accessand manipulate those objects. In the Outlook Object Model 462, theobject MailItem is an instance of an email; the object Inspector is aninstance of an Inspector window 401; the object Explorer is an instanceof an Explorer window 400; the object objExpMailItem is an emailselected by the user and currently being viewed in the preview pane 495of Explorer window 400; and the object objMailItem is an email selectedby the user and currently being viewed in an Inspector window 401.

Event Handler 462 in EmailDial Add-in 460 uses its access to OutlookObject Model 452 in MS Outlook 450 to listen for events that are firedby active objects in MS Outlook 450 at specific times: theSelectionChange event which is fired when an email is being loaded intopreview pane 495 of Explorer window 400; the NewInspector event which isfired when the user has selected (i.e., double-clicked) an e-mail in theExplorer window 400, thereby causing an Inspector window 401 to appear;and any event which requires that an email that may have been tagged bythe Email-Dial Add-in 460 to be restored to its original format, such asForward, Reply, ReplyAll, Write, etc.

When a SelectionChange or NewInspector event is detected, Parser 464parses the run-time Outlook Object Model of the email being loaded intoeither the preview pane 495 of Explorer window 400 or Inspector window401 using the Outlook Object Model API (much in the same way as the DOMAPI 252 in FIG. 2B of the first presently preferred embodiment).Specifically, Parser 464 parses the email object being loaded to findany instances of telephone numbers using regular expression parsingrules designed to pick up the various formats that phone numbers cantake within a computer document, in a manner similar to the first andsecond presently preferred embodiments. Although not shown in theembodiment of FIG. 4B, it is also possible to have a module such asClassifier 266 in FIG. 2B for further filtering of potential telephonenumbers.

Parser 464 outputs the detected telephone numbers to Markup Module 468,which will highlight and provide the GUI functionality for the detectedtelephone numbers. However, because the email may be in any format,including Plain text or RTF which do not allow markup tags, any emailwhich is being loaded for viewing must be converted into HTML format(thereby creating a provisional viewing computer document, as in step170 of FIG. 1C). When this conversion is made, a field is added to theprovisional viewing email which indicates the original format of theemail (e.g., Plain/RTF/HTML). In this way, the provisional viewing emailcan be converted back to its original format after viewing hascompleted, or before an action is performed with or upon the email.

Markup Module 468 creates a tag which is placed around the telephonenumber in the provisional viewing email (which is maintained in its ownnamespace), as in step 133 of FIG. 1C. As in the other presentlypreferred embodiments, the tags indicate that any detected phone numberwill be highlighted and active upon certain cursor events. Specifically,when the cursor is over an underlined telephone number, the “i” button420 will appear in the GUI window above the underlined telephone numberindicating that functionality is available for this number. If the “i”button is selected, the appropriate menu 430 of options appears. If the“Call Number” option is selected, Event Handler will call and providethe appropriate number string to IPSoftphone 470, which will perform thedialing and connecting.

When any event, such as Forward, Reply, ReplyAll, Write, etc., whichrequires that the provisional viewing email be restored to its originalformat, is detected (step 180 in FIG. 1C), the provisional viewing emailis stripped of any tagged telephone numbers and then restored to itsoriginal format according to the field added to the provisional viewingemail HTML document for such a purpose (step 190 in FIG. 1C).

The Email-Dial Add-in 460 ensures that the provisional viewing emailHTML document is created as late as possible (before viewing) and undoneas early as possible (before an action requiring the originallyformatted email is performed). Furthermore, the Email-Dial Add-in 460works transparently, and, in some embodiments, can be enabled/disabledby the user using a bookmarklet, preference setting, or other means.

Additional Functionality

Thus far, the additional functionality of a tagged telephone number hasbeen described in relation to my prior patent applications referencedabove. According to the embodiments disclosed herein, however, theinventive method provides for further functionality for the telephonenumbers uncovered by the inventive method, and found in the computerdocuments searched thereby.

Facsimile Transmissions

For example, if it is determined (e.g., by uncovered contextinformation), that a particular tagged telephone number is a facsimilenumber, it may be possible for the user to click directly on the taggedtelephone number, and send a facsimile transmission to that number. Theact of clicking may activate an applet to create a facsimile “coverpage”, in known fashion, as is common in most word processing programs,and to give the user the ability to append a further document, such asan already generated word-processing document, as part of the facsimiletransmission. The cover page may be generated automatically, in whole orin part, and may be based upon the context information, such as wherethe context information includes the name of the facsimile recipientand/or a proposed subject. In most cases, the user may be given theopportunity to modify the cover page, but, in some cases, such as wherethe computer document encourages the use of a standardized cover pageformat, the option to modify may be limited or eliminated.

Messaging

According to another aspect of the invention, the additionalfunctionality may be the ability to transmit a message directly to thetelephone number, as where the telephone number is associated with ashort message service (SMS), such as a pager, or an instant messaging(IM) account. In these cases, the invoking of the additionalfunctionality allows the user to send a suitable message to the taggedtelephone number. The message may be of a particular nature, such as arequest to initiate a telephone conversation immediately, or scheduleone at a later time. The telephone conversation being offered may alsobe a conference call, with the other participants identified, or not, atthe option of the user.

Additionally, the information sent via SMS or IM may be either input bythe user or captured from the document in which the number was found. Inthe case of an IM message, the tagged telephone number may be sent to acorporate server which returns back the IM address associated with thetagged telephone number, and that IM address is used to initiate an IMsession with the clicked telephone number.

Sending Telephone Numbers to Other Devices

A particular benefit of the inventive method is the ability to transmittelephone numbers to devices and applications associated with the user'sdevice and the specific application in which the number was uncovered.For example, if a user finds a telephone number of interest, it may besent to a separate device, such as a personal digital assistant (PDA),cellular telephone, a laptop or other device having a database ofcontact or other similar information. The device may be physicallydisconnected from the device on which the user displayed the GUI inwhich the telephone number was highlighted. In this case, the telephonenumber, together with any context information associated therewith, maybe sent to the separate device by any suitable means, such as Bluetoothor IrDA transmissions, in known fashion. These separate device mayreceive the transmission, and store the telephone numbers in knownfashion. Alternatively, such as where the separate device is a telephonedialer, the device may act immediately on the telephone number sotransmitted.

In situations, where the separate device is physically connected to theuser's device, but logically separate therefrom, the telephone numberand context information may be sent over a physical, or wiredconnection. This would be, for example, where the separate device isconnected to the user's device over a network (including, possibly, theInternet), or through a cable (such as, for example, in the case of asynchronization cable or USB connection), or even as different parts ofthe user's computer (such as, for example, different logical drives,different partitions, different folders, different programs or differentfiles). Each of these types of relationships are examples of “logicalconnections” and the like as used herein. Devices which are not“logically connected” are therefore “logically separate”.

It would also be possible, by means of the inventive method, to sendtagged telephone numbers and/or context information to more than oneseparate device, such as through a WAP push. This feature may be usefulin sending tagged telephone numbers and context information to a groupof cellular telephones which may require the tagged numbers and/orcontext information. Provided the cellular telephone are registered witha push server, the desired data may be transmitted to the server, andthen to a plurality of other devices registered therewith. This may beespecially useful in the case of sending updated contact information toa distributed work force, such as for salespersons or repairpersons inthe field.

Multiple Tagged Numbers

Many computer documents will contain multiple telephone numbers. Forease of handling these numbers, and a standardized format for presentingthem to the user, it may be desired to generate a separate window in theGUI in which all or some of the telephone numbers are displayed,possibly together with any relevant context information uncovered by themethod. Then, the user may be given the option of performing the samefunction with respect to all of the tagged and/or displayed numberssimultaneously, such as saving them all, setting up a conference callwith them all or sending a SMS or IM broadcast to all.

It is possible, however, that less than all of the tagged telephonenumbers so displayed may actually be of interest to the user. In thiscase, the user may be given the option of selecting specific ones of thetotal number of tagged telephone numbers, and deleting them from thedisplay. The user may then act on all desired tagged telephone numbers,and perform the same action upon all of the desired numbers. Forexample, by choosing relevant numbers from the list, the desiredtelephone numbers, together with any uncovered (and desired) contextinformation) may be sent to the user's PDA (as described above).

Checking the Status of the Tagged Telephone Number

Once a telephone number is tagged in a computer document, the user mayhave the option of attempting to establish communication with thatnumber as described previously. It is possible, however, that the partywith whom the tagged telephone number is associated may prefer to havecommunications under certain conditions (such as, between 9 am and 5 pm,Eastern Time, Mondays through Fridays), or by means of a specificpreferred means of communications (such as, preferably by IM). It isalso possible that the party associated with the tagged telephone numbermay be perfectly willing to receive the phone call from the user at thismoment, but for the fact that that party is already on the telephone.

In such instances, it would be useful if the user were told the statusof the tagged telephone number. For example, the number could be checkedto ensure it is still in service, has a forwarding number associatedtherewith, or is busy or malfunctioning. Once this status is determined,the user may be informed of the status of the line.

Alternatively, the context information could be checked to determine ifthe telephone number has any preferences associated therewith, such asthe time or day to be called, or any preferred mode of communication(possibly listed in a hierarchical list of preferences). All of thisinformation may be presented to the user for informing the user'sdecision on how to best utilize the information gathered from thetagging of the telephone number and the uncovering of the contextinformation.

Creation of Additional Context Information

As explained in my prior filed application Ser. No. 11/224,160,referenced above, context information is often useful for any taggedtelephone number. That information may be found in the computer documentitself, or generated or edited by the user into a form the user findsmost helpful. In some contexts, the context information may even begenerated during a communication session initiated with the taggedtelephone number. For example, the user may find a tagged telephonenumber, and immediately initiate a telephone conversation with the partyat the tagged telephone number. During the conversation, the user maywish to take notes, or record all or part of the conversation (by audioalone or in connection with a video call). This information may beuseful as context information. According to the method, a user may begiven the option of opening an application in which the tagged telephonenumber was not found, such as a video capture, audio capture, orword-processing program, and generate context information whileconnected the tagged telephone number. This context information may besaved and made part of the context information associated with thetagged telephone number. Alternatively, the context information could bein the form of digital information from another source, such as a savedphotograph or sketch generated by some third party and sent to the userby e-mail. All of these uses permit the saving of context informationgenerated before, during or after the communication session with thetagged telephone number.

Using Tagged Telephone Numbers and/or Context Information to AddFunctionality to Other Applications

According to a further aspect of the invention, a tagged telephonenumber may add functionality to applications other than the one in whichit was found.

For example, one common use of the invention is expected to be in thecontext of browsing web pages through a web browser, such as InternetExplorer (IE). In this application, when a telephone number isuncovered, it is tagged and highlighted in IE, and the user is thereforealerted to the possibility that there is a telephone number of interest,and then given the opportunity to send that telephone number and/orcontext information to another application, such as a contact manager.This is described in my prior pending applications.

This information may be of even further benefit, however, if theinformation is placed on the toolbar of the other application andprovided even further functionality. For example, if the user uncovers aweb page having a possible new tagged telephone number, the user maywish to know if the telephone number is already in the contact manager,and, if so, whether the context information already present in thecontact manager contains all of the context information found in thecurrent web page. This may be accomplished by adding a button to thetoolbar of the contact manager, and enabling the contact manager toreceive the telephone number and/or context information directly fromIE. The contact manager could then be enabled to search for theinformation sent to the toolbar, for example, to find other listings ofpersons from the same company, or other sources for spare parts, or anyother information that may provide a desired context for the user in aparticular application.

It would also be useful, in this environment, to permit the user to lookto see if there has been any prior contact with the telephone number,and review the context information, including any notes or comments withrespect thereto. This may be or particular utility where the database inwhich this information is stored is accessible to more than one user,and so not every user may know of any such prior contacts.

All of the implementation details and available options have not beendescribed in detail with reference to any particular embodiment, as theymay be gleaned from the descriptions in the '468 application and thedescriptions of the other presently preferred embodiments describedherein.

While there have shown and described and pointed out fundamental novelfeatures of the invention as applied to a preferred embodiment thereof,it will be understood that various omissions and substitutions andchanges in the form and details of the method steps described, thedevices illustrated, and the operation thereof, may be made by thoseskilled in the art without departing from the spirit of the invention.For example, it is expressly intended that all combinations of thoseelements and/or method steps which perform substantially the samefunction in substantially the same way to achieve the same results arewithin the scope of the invention. Moreover, it should be recognizedthat structures and/or elements and/or method steps shown and/ordescribed in connection with any disclosed form or embodiment of theinvention may be incorporated in any other disclosed or described orsuggested form or embodiment as a general matter of design choice. It isthe intention, therefore, to be limited only as indicated by the scopeof the claims appended hereto.

1. A method for automatically detecting instances of telephone numbersin a computer document and for providing functionality for any detectedinstances of telephone numbers in the computer document, comprising thesteps of: when opening a computer document in an application such thatthe computer document will be rendered in a graphical user interface(GUI) window: creating a provisional markup language document forvisually rendering the computer document in the GUI window;automatically parsing the computer document and/or the provisionalmarkup language document for any instance of a telephone number; andplacing tags around any detected instance of a telephone number in theprovisional markup language document; visually rendering the provisionalmarkup language document in the GUI window such that any taggedtelephone number provides primary visual indicia that (i) it is atelephone number, and (ii) functionality is available for the telephonenumber; and providing functionality for a user to select any taggedtelephone number in order to perform or initiate one or more desiredoperations either with or upon the selected tagged telephone number,wherein said functionality is not native to the computer document;wherein the functionality is selected from the group consisting of:sending a facsimile transmission to the tagged telephone number; sendinga message to the tagged telephone number; and sending the taggedtelephone number to a device distinct from the device used by the userto parse the computer document.
 2. The method of claim 1, wherein theselected functionality is sending a message to the tagged telephonenumber.
 3. The method of claim 2, wherein the message is a short messageservice (SMS) message
 4. The method of claim 2, wherein the message isan instant message (IM).
 5. The method of claim 4, wherein the IM issent to a corporate server, which returns an address to the user toinitiate an IM session with the tagged telephone number.
 6. The methodof claim 2, wherein the message is input by the user.
 7. The method ofclaim 2, wherein the message contains information captured from thecomputer document in which the tagged telephone number is found.
 8. Themethod of claim 2, wherein the message is a request to schedule acommunication session.
 9. The method of claim 2, wherein thecommunication session is a telephone call with the tagged telephonenumber.
 10. The method of claim 9, wherein the telephone call is aconference call involving at least the tagged telephone number.
 11. Themethod of claim 1, wherein the selected functionality is sending thetagged telephone number to said distinct device from said device used bysaid user.
 12. The method of claim 11, wherein said distinct device isphysically separate from said device used by said user.
 13. The methodof claim 12, wherein said distinct device is a cellular telephone. 14.The method of claim 12, wherein said distinct device is a personaldigital assistant (PDA).
 15. The method of claim 12, wherein said taggedtelephone number is sent by means of a wireless transmission.
 16. Themethod of claim 15, wherein said tagged telephone number is sent bymeans of an infrared data (IrDA) transmission.
 17. The method of claim15, wherein said tagged telephone number is sent by means of a Bluetoothtransmission.
 18. The method of claim 15, wherein said distinct devicecomprises more than one separate distinct device.
 19. The method ofclaim 18, wherein said tagged telephone number is sent by means of a WAPpush.
 20. The method of claim 1, wherein there is more than one taggedtelephone number in the computer document.
 21. The method of claim 20,further comprising the step of opening a window in which said more thanone tagged telephone number is displayed.
 22. The method of claim 21,further comprising the step of permitting the user to perform one ormore actions on at least one tagged telephone number in the window. 23.The method of claim 22, wherein one of said actions is saving said atleast one tagged telephone number.
 24. The method of claim 22, whereinone of said actions is establishing a conference call with said at leastone tagged telephone number.
 25. The method of claim 22, wherein one ofsaid actions is broadcasting a message to said at least one taggedtelephone number.
 26. The method of claim 20, further comprising thesteps of identifying context information surrounding each said taggedtelephone number and displaying said context information in conjunctionwith its respective telephone number.
 27. The method of claim 1, furthercomprising the step of checking the status of the tagged telephonenumber.
 28. The method of claim 27, wherein said status includes apreferred mode of communication with said tagged telephone number. 29.The method of claim 28, wherein said functionality further comprisesgiving the user the option of communicating with said tagged telephonenumber by said preferred mode of communication.
 30. The method of claim1, further comprising the step of opening another application while incommunication with said tagged telephone number to permit the creationof context information which may be associated with said taggedtelephone number.
 31. The method of claim 30, wherein said contextinformation is text information.
 32. The method of claim 30, whereinsaid context information is video data.
 33. The method of claim 30,wherein said video data is video-captured during a video communicationsession with said tagged telephone number.
 34. The method of claim 30,wherein said context information is audio data.
 35. The method of claim34, wherein said audio data is recorded during a communication sessionwith said tagged telephone number.
 36. The method of claim 30, whereinsaid context information includes digital information generatedindependently of the communication session.
 37. The method of claim 30,further comprising the step of associating data stored through saidapplication with said telephone number.
 38. The method of claim 30,wherein said step of associating data includes associating said saveddata with said tagged telephone number automatically when saved in saidopened application during a communication session initiated through themethod.
 39. The method of claim 30, further comprising the step ofpermitting the user to save data at a later date and associate suchsaved data with said tagged telephone number as context information foruse at a later time.
 40. The method of claim 30, further comprising thestep of permitting the user to associate pre-existing contextinformation with a tagged telephone number.
 41. The method of claim 30,further comprising the step of permitting the user to associatelater-generated context information with a tagged telephone number. 42.The method of claim 1, further comprising the step of checking adatabase associated with the tagged telephone number for contextinformation.
 43. The method of claim 42, further comprising the step ofpermitting the user to retrieve said context information from saiddatabase, and associating said context information with said taggedtelephone number.
 44. The method of claim 1, wherein the selectedfunctionality is adding the tagged telephone number to a toolbar of anapplication other than one in which the step of parsing is performed.45. The method of claim 44, wherein said other application includes atleast one of the following options: permitting the user to search fortelephone numbers in a database; permitting the user to search forspecified context information in said database; automatically searchingsaid database for said tagged telephone number; and automaticallysearching for specified context information in said database.
 46. Themethod of claim 45, wherein said other application includesautomatically searching for said specified context information in saiddatabase and said method further comprises the step of displaying anytelephone number associated with an instance of said specified contextinformation in said database.
 47. The method of claim 1, wherein theselected functionality is sending a facsimile transmission to the taggedtelephone number.